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We first study labeled transition systems with explicit successful termination. We establish the no- 
tions of strong, weak, and branching bisimulation in terms of boolean matrix theory, introducing 
thus a novel and powerful algebraic apparatus. Next we consider Markov reward chains which are 
standardly presented in real matrix theory. By interpreting the obtained matrix conditions for bisimu- 
lations in this setting, we automatically obtain the definitions of strong, weak, and branching bisimu- 
lation for Markov reward chains. The obtained strong and weak bisimulations are shown to coincide 
with some existing notions, while the obtained branching bisimulation is new, but its usefulness is 
questionable. 

1 Introduction 

(Labeled) transition systems are a well established formalism for modeling of the qualitative aspects 
of systems, focusing on the behavioral part. A transition system is a directed graph in which nodes 
represent states of the system, and labels on arrows represent the actions that the system can perform 
when going from one state to another. One state is considered to be the starting state. It is sometimes 
useful to distinguish deadlock behavior (inability to proceed) from successful termination, so we consider 
transition systems in which some states are explicitly marked as (successfully) terminating ITBI ITII. 

Reasoning about transition systems is usually done by relating them according to some behavioral 
equivalence. If two systems are to agree on every step they take, then they are equivalent modulo strong 
bisimulation equivalence ifTSl |T6l . When a system can perform internal (silent) steps, of which the 
impact is considered unobservable, strong bisimulation is less appropriate because it equates too few 
states. To solve this problem weaker equivalences have been introduced that abstract away from the 
internal steps but require that the other, i.e. visible, steps are strongly simulated. The two most commonly 
used equivalences of this type are weak bisimulation [,16,1 and branching bisimulation SO equivalence. 
The difference between the two is that the latter preserves the branching structure of a transition system 
better ||9l. 

While transition systems are very useful for qualitative reasoning, (continuous-time) Markov chains 
have established themselves as powerful, yet fairly simple models for performance evaluation, i.e., for 
the modeling of quantitative behavior of systems. A Markov chain can be represented as a directed graph 
in which nodes represent states and labels on the outgoing arrows determine the stochastic behavior (an 
exponentially distributed delay) in the state. Some states are marked as starting and have initial proba- 
bilities associated with them. To increase modeling capability and obtain some very useful performance 
measures, such as throughput and utilization of a system, Markov chains are often equipped with re- 
wards ifTTl . We only consider rewards associated to states, in which case they actually represent the rate 
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with which a Markov chain gains a reward while residing in a state. A Markov chain with (state) rewards 
is called a Markov reward chain. 

The idea of strong bisimulation exists in the Markov reward chain world as an aggregation method 
called ordinary lumping This method is based on joining states that have the same reward 

and that transfer to lumping classes with equal probabilities at any given time. This ensures that the 
stochastic behavior of strongly bisimilar states is the same, and that bisimulation keeps (the reward based) 
performance properties. The notions of weak and branching bisimulation, however, have not yet been 
introduced to pure Markov chains (one exception is the weak bisimulation of lH that actually coincides 
with ordinary lumping) but only to their extensions coming from stochastic process algebras lITOl . All 
these extensions add action information to Markov chains, and, like in transition systems, take the special 
action T for the internal step that can be abstracted from. The abstraction is not stochastically formalized, 
in the sense that it is not defined what performance properties are shared among weakly bisimilar chains. 
Moreover, when restricted to exponential transitions, the existing equivalences typically coincide with 
strong bisimulation. 

In this paper we establish the notion of strong, weak, and branching bisimulation for Markov chains 
by setting the theory of transition systems and Markov chains on a common ground. The well developed 
matrix apparatus has shown to be a powerful method for presenting and reasoning about Markov chains, 
so we take matrix theory for the unifying framework. The three notions of bisimulation on transition 
systems are first formalized in (boolean) matrix terms, leading to a system of matrix equalities. These 
equalities can be directly interpreted in the (standard matrix) setting of Markov chains, automatically 
yielding definitions of strong, weak, and branching bisimulation there. The obtained notion of strong 
bisimulation is proven to indeed coincide with the standard definition of ordinary lumping. The obtained 
notion of weak bisimulation is shown to remarkably coincide with the notion of T-lumping that we have 
recently developed as a helping tool in solving a different and independent problem lfT5ll2Tl . To the best 
of our knowledge, the obtained definition of branching bisimulation does not correspond to any known 
Markovian equivalence from the literature. 

With the decision to use matrix theory as a common setting, the gain is twofold. The matrix approach 
to transition systems sets the theory in a powerful algebraic setting that can be used as an alternative to or 
in combination with the standard process algebraic approach. This is specially useful because the notion 
of (bi)simulation has been, in some forms, extensively studied in graph, modal logic, and automata 
theory [201181151. Matrices, moreover, increase clarity and compactness, simplify proofs, make known 
results from linear algebra directly applicable, have didactic advantage, etc. 

The structure of this paper is as follows. In the next section we give some preliminaries for working 
with matrices. In Section[3]we define transition systems with explicit termination as systems of matrices. 
We also give matrix definitions of strong, weak, and branching bisimulation, and show that these notions 
indeed correspond to the standard ones. In Section |4] we first standardly present the theory of Markov 
reward chains using matrix theory. Then we interpret the definitions from Section [3] in this setting and 
discuss the resulting notions of bisimulation. The last section gives some conclusions and directions for 
future work. 

2 Preliminaries 

Let X be a set with addition, multiplication, the unit elements and 1 for these operations, and with a 
preorder <. Then X"^™ denotes the set of all « x m matrices with elements in X. We assume that matrix 
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addition, matrix multiplication, multiplication by a scalar, and <, are all standardly defined inX"^'". The 
elementwise product of two matrices is defined as {AnB)[i,i] = A[i,i]B[i,i]. 

Elements of X''^" and are called (row and column) vectors. 1" denotes the vector in X"^' that 
consists of n I's. 0"^'" denotes the « x m matrix consisting entirely of zeroes. /" denotes the nxn identity 
matrix. We omit the n and m when they are clear from the context. A matrix A of which every element 
is either or 1, i.e. an element of {0, i}"^™^ is called a 0-1 matrix. 

A 0-1 matrix V G x"^^, n>N in which every row contains exactly one 1 is called a collector. For 
the theory of bisimulation the central notion is of partitioning of states into equivalence classes. We can 
then think of a collector matrix as a matrix in which the rows represent states, the columns represent the 
equivalence classes, and the entries indicate which states belong to which classes. Note that V l = 1. A 
matrix U G X^""' such that U - l = I and UV = is a distributor for V. 

3 Transition Systems in Matrix Terms 

Let A be a set and let ^(A) be the set of all subsets of A. Then P(A) = (^(A), + , •, ",0, 1) is a boolean 
algebra with +, •, ~, and 1 representing union, intersection, complement, the empty set and the full set 
A respectively. We use +, •, and 1 instead of U, fl, and A to emphasize the connections with standard 
matrix theory. 

We now assume that A is a set of actions and fix it for the reminder of this section. A transition 
system is standardly defined as a quadruple {S, —>-,So,l) where 5 is a finite set of states, ^ C 5 x A x 5 
is the transition relation, G 5 is the initial state and |C 5 is the set of (successfully) terminating states. 

In matrix terms we define a transition system as a triple of a 0-1 row vector that indicates which of 
the states is initial, a matrix whose elements are sets of actions that the system performs when transiting 
from one state to another, and a 0-1 vector that indicates which states are terminating. 

Definition 3.1 (Transition system) A transition system (of the dimension n) is a triple (a,A,p) where: 

• a G {0, 1}^^" is the initial vector with exactly one non-zero entry, 

• AG P(A)"^" is the transition matrix, and 

• p G {0, 1}"^' is the termination vector. 

The set of all transition systems of the dimension n is denoted =5^. □ 

If 5 = {^i , . . . , Sn}, our definition is obtained from the standard one by putting: 



That is, for each two states si and sj, A[i,j] contains the set of actions that the system can perform by 
going from to sj. The i-th element of a is 1 if the state Si is initial. The i-th element of p is either or 
1 depending if the state Si is terminating or not. It is clear that, given an ordered set S, we can obtain the 
standard definition from our definition easily. Figure [T] depicts a transition system and gives its matrix 
representation. The set of states is 5 = {si,S2,ss,S4}. State si is the initial state; states si and S4 are 
terminating. 
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Figure 1 : Transition system and its matrix representation 
3.1 Strong bisimulation 

Strong bisimulation is an equivalence relation that partitions the set of states in such a way that the set of 
actions that can be executed to reach some class is the same for every two states in a class. In addition, 
the termination behavior of two related states must be the same. This allows us to built the quotient (i.e. 
the lumped) system in which states are the equivalence classes. In matrix terms bisimulation conditions 
and the lumped system are conveniently expressible in terms of a collector and a distributor matrix. 
Definition 3.2 (Strong bisimulation-LTS) A collector matrix V G {0, 1}"^^ is called a strong bisimu- 
lation on the transition system (c7,A,p) G if 

VUAV = Ay and yf/p = p , 

where JJ is some distributor for V. In this case we also say that (c7,A,p) strongly lumps (by V) to the 
transition system (d',A,p) G 5^ defined by: 



c = cV, A = UAV and p=Up. 



□ 



Both the conditions for bisimulation and the definition of the lumped process do not depend on the 
particular choice of a distributor. Suppose that U'V = I for some U' ^ U. Then VU'AV = VU'VUAV = 
VUAV =AV and similarly VUp = p. Also U'AV = U'VUAV = UAV and similarly U'p = Up. 

We now show that our definition of strong bisimulation agrees with the standard one. Define R = VV^ 
and note that V^ is a also distributor for V. Clearly R>I. We first prove that the above conditions are 
equivalent to the conditions /M < A/? and /?p < p . We calculate RA = VV'^A < VV'^AVV'^ = AVV'^ = AR 
and Rp = VV'^ p < p, and for the other direction, AV < VV'^AV < AVV'^V = AV and p < VV'^ p < p. 
Note now that a G {RA)[i,j] iff there is a ^ such that R[i,k] = 1 and a eA[k,j]. Similarly, a G {AR)[i,j] 
iff there is an £ such that a G A[/, i] and R[£, j] = 1. The condition RA < AR then says that 



imphes 



This clearly corresponds to the standard definition of strong bisimulation. Finally, note that {Rp)[i] 
iff there is a j such that R[i,j] = 1 and p [j] = 1. Thus, the condition Rp < p says that: 



implies 
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This again matches with the standard definition. 
3.2 Weak bisimulation 

A silent step in a transition system is a step that is labeled by the internal action T. Every matrix T G 
P(A)"^" can be uniquely represented as T = A + {t} • S where T € A, and A,S e P(A)"^" are such that 
{t} • A = and 5 is a 0-1 matrix. To make this form of T more explicit we write {a,A,S,p) instead 
of {a,T,p). Note that the strong bisimulation conditions from the previous section can be decomposed 
into separate conditions on A and S. In other words, the condition VUTV = TV is valid if and only if the 
inequalities WAV =AV and VUSV = SV both hold. 

Weak bisimulation |16 | ignores silent transitions in a very general way. It requests that a transition 
labeled with an action is simulated by a transition labeled with the same action but preceded and followed 
by a sequence of T transitions. For this we introduce a matrix definition of reflexive-transitive closure. 
Given a 0-1 matrix R e {0, 1}"^", we call the matrix R* = L,7=o^" the reflexive-transitive closure of R. 
Definition 3.3 (Weak bisimulation) A collector matrix V G {0, l}"^'^ is a weak bisimulation on the 
transition system (a,A,5,p) G 5^' iff 

vunv = nv,vuuAnv = nv, and vunp = up 

where where IT = 5* is the reflexive transitive closure of S and U is some distributor for V. We say that 

{a,A,S,p) weakly lumps (by V) to {a,A,S,p) defined by 

d = oV, A = V'^AV and p = V'^p. □ 

Contrary to strong bisimulation the definition of the lumped process now depends on the distributor 
used. Using any other distributor would, in general, give a different result for the lumped system (the 
irrelevance of distributors is only implied by the strong bisimulation condition, i.e. by WAV = AV and 
VUp = p, which might not hold here). 

Our definition of weak bisimulation corresponds to the standard one. First, S*[i,j] = 1 iff there is an 
n>0 such that 7] = 1. This is equivalent to saying that there exist /q; • • • such that /q = /, /„ = j 
and S 4+ 1 ] = 1 for all ^ = 1 . Recall that S[i,j] = l means , in the standard theory, that Si —i- sj . 
Thus, n[/,7] = S*[i,j] = 1 means that we have Si^^ . . 5,-,, or that, in the standard notation, Sj =^ sj. 
Now, as we did for strong bisimulation, we let R = VV'^ and express the bisimulation conditions using R. 
We have RS<Rn< RUR = VV'^UV = UVV'^ = UR,RA< RUAUR = VV'^UAUVV'^ = UAUVV'^ = 
nAnT?, and Rp < RYlp < Yip. The first inequality means that 

Si- 
implies |, 

se- - - Sj. 

For the second inequality note that a G {RA)[i,j] iff there is a ^ such that R[i,k] = 1 and a G A[^,j]. 
Now, a G {UAUR)[i,j] iff there exist 1 < e,f,t < n such that n[i,e'] = 1, aeA[£',i"], n[i",i] = 1 and 
R[iJ] = l. Therefore, RA < 114117? means that 



implies 



1- 

*7 



se - 
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for a ^ T. Finally, Rp < Tip means that 



implies 



Si 

I. 

sel 



This is the standard definition of weak bisimulation (with explicit termination). 

Weak bisimulation can also be interpreted as a strong bisimulation on a system closed under the 
sequence of t transitions, inducing the following diagram: 



Transition System ■ 



z-ctosure 



weak 
lumping 



Weakly Lumped 
Transition System 



T-closed 
Transition System 



We show that weak lumping is sound in the sense that also 



Weakly Lumped 
Transition System 



z-closure 



induced 
strong lumping 



Strongly Lumped 
T-closed Transition System. 



Strongly Lumped 
T-closed Transition System. 



The main purpose of the proof is to illustrate the power of matrices in this setting. We first prove two 
important properties of . 

Theorem 3.4 V'^UV = {V'^SV)* and UVV'^ = UVV'^U. □ 
Proof We have V'^UV = (l„>o5) V <V'^ (l„>()5VV'^) V = I„>o V^^V = {V'^SV)* and UVV^ < 

nvv'^n < nvv'^nvv'^ = nnvv'^ = nvv'^. m 

Using this theorem and the conditions of Definition [33] we calculate {V'^SV)*V'^AV{V'^SV)* = 

v'^nvv'^Avv'^nv = v'^nvv'^Anv = v'^nvv'^UAnv = v'^nuAUv = v'^uauv and v'^np = v'^nnp 

y"'"nVV"''np = y"'"nyy"'"p = {V'^SVyv'^p, which exactly states that the order of application of T- 
closure and lumping is irrelevant. 



3.3 Branching bisimulation 

Branching bisimulation f9] preserves the branching structure of a system more than weak bisimulation 
by requiring that after the initial sequence of T steps the resulting state must again be bisimilar to the 
same state that the starting state is bisimilar to. 

Definition 3.5 (Branching bisimulation) A collector V € {0,1}"^^ is a branching bisimulation on 
{a,A,S,p)e^^ iff 

y[/(/+n(^)5)y = (/+n(^)5)y,v[/n(^Uv = n(^Uv, and vt/n^^'p = n(^'p 

where where n'^' = {SnVV'^y, and U is some (any) distributor for V. We say that (a, A, 5, p) branching 
lumps (by V) to {a,A,S,p) defined by 

a = oV, A = V'^AV and p=V'^p. □ 
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Note that riy = {SnVV'^)* <S* =S*{I + S), showing the known result that every branching bisimulation 
equivalence is also a weak bisimulation. 

To show that our definition indeed induces the notion of branching bisimulation, we again let R = 
V'^V. The conditions of Definition 13.51 are easily shown to be equivalent to the conditions RS < R + 
RA < n^^^AR, and Rp < Tl^^^p, which are in turn easily shown to exactly match the standard 
conditions of branching bisimulation. 

Similarly as we did for weak, we can interpret branching bisimulation as a strong bisimulation on a 
system closed under the sequence of T-transitions that now connect states from the same class only (note 
that the closure then depends on the bisimulation). This induces the following diagram: 



Transition System ■ 



branching 
lumping 



Branchingly Lumped 
Transition System 



zy -closure 



T,y-closed 
Transition System 



induced 
strong lumping 



Strongly Lumped 

TjV-closed 
Transition System. 



As it was the case for weak bisimulation, the diagram can be closed, i.e. 



Branchingly Lumped 
Transition System 



xj-closure 



Strongly Lumped 

T,y-closed 
Transition System. 



Since {V^ SV)* □ / = /, this amounts to showing ih&tl + V^SV = V'^ {S* nR){I + S)V, V^AV = {S* U 
/?)AV and V^p = V'^ [S* nR)p which easily follows from the conditions of Definition [3311 



4 Markov Reward Chains 

We now turn to Markov reward chains. We define the notions of strong, weak and branching bisimulation, 
by directly interpreting the conditions of the previous section in the real-number matrix setting. For each 
new notion we discus how it relates to some exiting reduction technique for Markov reward chains. 

A Markov chain is a time-homogeneous finite-state stochastic process that satisfies the Markov prop- 
erty (future independent of the past). It is completely determined by a stochastic transition matrix (func- 
tion) P{t), holding the probabilities of being in some states at time f > 0, if at a given state at time 0, and 
a stochastic row vector that gives the starting probabilities for each state. 

The matrix P{t) can conveniently be expressed in terms of a time-independent generator matrix. 
A generator matrix is a square matrix of which the non-diagonal elements are non-negative and each 
diagonal element is the additive inverse of the sum of the non-diagonal elements of the same row. The 
elements of this matrix are exponential rates It is a standard Markov chain result that for every P{t) there 
exists a unique generator Q such that P{t) = e^' (and then also Q = P'{0)). 

A Markov reward chain is a Markov chain where reward is associated to every state, representing the 
rate at which gain is received while the process is in that state. We now give a formal definition. 



^We are not aware that this resuU has been obtained before, although the corresponding one for weak bisimulation is known. 
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Definition 4.1 (Markov reward chain) A Markov reward chain is a triple {o,Q,p) G M^^" x M"^" x 

M"^ 1 where a is the initial probability vector, 2 is a generator matrix called the rate matrix, and p is the 
reward vector. □ 



l-TT 




(7 = (;r l-;r 0) 



Q 




P = 



Figure 2: A Markov reward chain and its matrix representation 

Figure [2] gives an example of a Markov reward chain. This chain starts from state with probability 
71 and from state S2 with probability I — n. In state si it waits the amount of time determined by the 
minimum of two exponentially distributed delays, one parameterized with rate A , the other with rate jj. 
(note that this means that the process spends in state 1 exponentially distributed time with rate A + /i). 
After delaying the process jumps to state ^2 or state depending on which of the two delays was shorter. 
In state S2 the process just stays forever, i.e. it is absorbed there. From state S3 it can jump back to state 
s\ , after an exponential delay with rate V. While residing in state 5,, for / = 1 , 3, the chain earns a reward 
with rate r, . 

If in Definition |3.2| of strong bisimulation for transition systems we replace the transition system with 
a Markov reward chain {a,Q,p), we obtain the following definition of strong bisimulation for Markov 
reward chains. 

Definition 4.2 (Strong bisimulation-MRC) A collector matrix V G {0, 1}"^^ is called a strong bisim- 
ulation on the Markov reward chain {a,Q,p) iff 

VUQV = QV and VUp=p, 

where U is some (any) distributor for V. In this case we also say that {a,Q,p) strongly lumps to the 
Markov reward chain {a,Q,p) € defined by: 

d = oV, Q = UQV and p = Up. □ 

While the classical strong bisimulation requires that bisimilar states go to the same equivalence class 
by performing exactly the same actions, here these states must have equally distributed waiting times 
and equal joint probabilities when transiting to other classes. Moreover, they must also have the same 
reward. The definition reveals the already known fact that strong bisimulation corresponds to the notion 
of ordinary lumpability for Markov reward chains. The conditions from above exactly match the lumping 
conditions proposed in [ 17 1. Standard lumping is known to preserve many useful performance properties. 
For example, the total reward rate at t, defined as R{t) = oP{t)p, is easily shown to be the same for the 
original and the lumped chain. 

We now define the notion of weak bisimulation for Markov reward chains. Similarly to transition 
systems we introduce internal steps in a Markov reward chain by assuming that Q is of the form Qs + TQf, 
for some (fixed) parameter T > and two generator matrices Qs and Qf. To indicate this form of Q we 
write (a, Qs, Qf,p) for such a Markov reward chain. 
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To be able to apply the classical definition of weak bisimulation to Markov reward chains we must 
first find a matrix IT that would correspond to the notion of reflexive-transitive closure. It is not hard to 
see that this IT must be the ergodic projection at zero of Qf, defined by IT = limj^ooe^^' = Y.n=oQf^. 
(the strict formalization of this fact would need to be based on the theory of eigenvectors for boolean 
matrices). The matrix n always exists, and is a stochastic matrix denoting the probabilities that the chain 
occupying some state is found in (other) some state in the long run. 

Now, as we did for strong bisimulation, putting (a, Qs, Qf,p) instead of {a,A,S,p) in Definition [331 
we obtain the following definition of weak bisimulation for Markov reward chains. 

Definition 4.3 (Weak bisimulation-MRC) A collector matrix V E {0, i}"^^ is called a weak bisimula- 
tion on the Markov reward chain ((7, Qs, Qf,p) if 

vunv = nv,vunQ,nv = nanv and vuup = up, 

where n is the ergodic projection of Qf, and U is some distributor for V. □ 

We now explain what this weak bisimulation means stochastically. It is a known result from per- 
turbation theory that P-rit) = e^' = e'S^+^G/)' (uniformly) converges to Yle^^'^' when T ^ oo, where 11 
is the ergodic projection at zero of Qf. The matrix Yle^^'^' is a non-standard transition matrix |6l be- 
ing discontinuous at t = 0. It is, however, a transition matrix of a discontinuous Markov chain Q, a 
stochastic process that generalizes standard Markov chains by being allowed to perform infinitely many 
transitions in finite time. Strong bisimulation easily extends to discontinuous Markov reward chains by 
adding the condition VUIIV = YIV to Definition 14.21 Weak bisimulation can then be interpreted as a 
strong bisimulation on the discontinuous Markov reward chain that is obtained when T —> oo. The idea 
of taking the limit is to treat the transitions from Qf as instantaneous whenever we abstract from them; 
if a transition takes time it must be considered observable. This exactly was our motivation in ifTSl 1211 
where we defined the notion of T-lumpability, and it explains why the conditions for weak bisimulation 
exactly match the conditions of T-lumpability. 

The concept of weak lumping, i.e. of reduction modulo weak bisimulation, can also be introduced 
here. However, while we were able to use V'^ for transition systems as a special distributor ensuring that 
the order of application of T-closure and lumping is irrelevant, here a more complicated analysis is needed 
(in the real-number matrix theory, V'^ is not even a distributor!). One of the main results from iTSlflTI 
is the notion of a T-distributor W that is used to define the lumped process as {aV,W QsV,W QfV,W p). 
This is a special distributor that gives a lumped chain of which the limit is the lumped version of the limit 
of the original chain. In other words, it ensures that the following diagram commutes: 

Markov Reward Chain ^ Discontinuous 

with Fast Transitions t^oo Markov Reward Chain 



T-lumping 



ordinary 
lumping 



T-lumped lumped 

Markov Reward Chain ^ Discontinuous 

with Fast Transitions Markov Reward Chain 

The precise definition of W is complicated and outside the scope of this paper; it can be found in 1211 
together with the proof of the above diagram. Note that although a T-distributor W and the special 
distributor V'^ from the transition system setting appear to have no connections at all, they actually 
represent the same thing. In |[2ll we have shown that W is a distributor that satisfies nVlVIT, and is such 
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that the ergodic projection of WQfV is WnV . Recall that these are exactly the properties of V'^ that we 
established in Theorem l3.4[ interpreted in the boolean matrix setting. 

Having defined the weakly lumped process, we can speak of properties that are preserved by lumping. 
It can, e.g., be shown that the expected reward at t is the same for the two chains in the limiting case of 
T. 

To define branching bisimulation for Markov reward chains we first note that Qf □ VV'^ is in general 
not a generator matrix; the diagonal of Qf is never affected by this operation. We will, however, conve- 
niently assume that the obvious small adaptation has been made on the diagonal of Qf to turn Qf □ VV'^ 

into a generator, and we will denote the obtained matrix . 

Putting (a, Qs, Qf,p) instead of {a,A,S,p) in Definition l3.5l and using the cancelation property valid 
in M"^", we obtain the following definition of branching bisimulation for Markov reward chains. 

Definition 4.4 (Branching bisimulation-MRC) A collector matrix V G {0, l}"^^ is called a branching 
bisimulation on the Maitov reward chain {(J,Qs,Qf,p) if 

vun^^^QfV = nvQfV, vun'^^^av = n'^^^av, and y[/n(^V =n(^'p, 

where n'-^-' is the ergodic projection of Qf^K and U is some distributor for V. □ 

Note that there is actually no branching structure to be preserved in Markov reward chains as (almost) 
every t, being instantaneous in the limit, would have priority over any regular (exponential) transition. 
This makes the usefulness of branching bisimulation in this setting questionable. Moreover, in contrast 
to transition systems theory, we have not been able to prove that the above definition ensures that ev- 
ery branching bisimulation is also a weak bisimulation, nor whether there exists a commuting diagram 
similar to the one for strong and weak lumping. 

5 Conclusions and Future Work 

We used matrix theory as a unified framework to present the theory of transition systems and Markov 
reward chains, and of their bisimulations. The notions of strong, weak, and branching bisimulation on 
transition systems were first coded in terms of matrix equalities. The compactness and the algebraic 
power of this representation is then illustrated in few important theorems. The same matrix equalities 
were next interpreted in the Markov reward chain setting, directly leading to the notions of strong, weak, 
and branching bisimulation there. The obtained notion of strong and weak bisimulation were shown to 
coincide with the existing notions of ordinary and T-lumpability respectively. The obtained notion of 
branching bisimulation is new, but its properties are unknown and its usefulness is yet to be seen. 

In ll2n [T4i another form of aggregation in Markov reward chains is presented. In contrast to lumping 
this method always eliminates all internal steps by allowing states to be split into multiple classes. For 
future work we schedule to investigate whether this new reduction would lead to an interesting notion in 
the transition system setting. We also plan to see if the bisimulation-up-to technique |[T9i (formalized in 
matrix terms in |[2Ti ) is applicable to Markov chains. 
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